A Compilation of Annotated Bibliographies

This paper extends prior work on knowledge consolidation and the stability-plasticity problem within the context of a Lifelong Machine Learning (LML) system. A contextsensitive multiple task learning (csMTL) neural network is used as a consolidated domain knowledge store. Prior work has demonstrated that a csMTL network, in combination with task rehearsal, can retain previous task knowledge when consolidating a sequence of up to ten tasks from a domain. However subsequent experimentation has shown that the method suffers from scaling problems as the learning sequence increases resulting in the loss of prior task accuracy and a growing computational cost for rehearsing prior tasks using larger training sets. A solution to these two problems is presented that uses a sweep method of rehearsal that requires only a small number of rehearsal examples (as few as one) for each prior task per training iteration in order to maintain prior task accuracy.